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Abstract 


School failure often begins in the early grades. 
Research establishes the importance of early reading 
and math skills in predicting both academic success’ 
and grade retention. In two studies, we examined the 
impact of ABCmouse.com Early Learning Academy, 
a comprehensive supplemental digital early learning 
resource, on the acquisition of literacy and math skills in 
a longitudinal study of prekindergarten and kindergarten 
students. In Study 1, all participating prekindergarten 
students were identified as at risk for school failure. 
Those students who completed more ABCmouse learning 
activities evidenced greater academic gains than those 


who completed fewer activities. 


In Study 2, we followed the prekindergarten sample into 
kindergarten with a quasi-experimental design. Study 
2 demonstrated that regular ABCmouse usage helped 
accelerate kindergarten growth in literacy and math 
across multiple assessments. Even students who began 
with a lower readiness level at kindergarten entry were 
able to catch up to their peers through regular usage of 
ABCmouse. 


Results suggest that regular usage of ABCmouse 
across two grade levels helped students who had been 


identified as at risk for school failure to better prepare for 


kindergarten and achieve greater growth in literacy and 
math during kindergarten. 


Participants 


Study 1 (2013—14) included 230 prekindergarten students 
(49% male, 51% female) from 12 classrooms in the public 
school district prekindergarten (DPK) program in Tupelo, 
Mississippi. Enrollment was prioritized for children whose 
pretest scores indicated a risk for school failure, as well as 
English language learners, children with developmental 
disabilities/delays, and/or children living in poverty or 
homelessness. 


Study 2 (2014-15) included 571 kindergarten students; 
210 were students from the DPK sample, and 361 students 
had not taken part in the district prekindergarten program 
(non-DPkK). Four elementary schools (33 classrooms) 
participated, with eight or nine kindergarten classrooms 
of 12 to 20 students in each school. 


Design & Procedure 


Study 1 had a naturalistic design in which DPK students 
had varying usage of ABCmouse during the school year. 
The school district tested students at the beginning and 
end of the school year with the nationally validated Early 
Prevention of School Failure (EPSF) assessment.? 
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Study 2 used a quasi-experimental design, in which the 
district assigned two elementary schools full access 
to ABCmouse for the entire school year (Full-access 
Group). The other two schools in the district received 
restricted access to ABCmouse, including no access to 
most ABCmouse literacy activities for the first half of the 
year, and full access to the entire ABCmouse curriculum 
for the second half (Restricted-access Group). Teachers 
were provided training and instructed to use ABCmouse 
for at least 45 minutes per week with their students. The 
students accessed ABCmouse in various ways during 
the kindergarten year, including tablet carts, computer 
labs, and classroom computers and tablets. As a result of 
the research design, the Full-access Group not only had 
greater access to the full ABCmouse curriculum but also 
completed many more ABCmouse learning activities, 
especially literacy activities, than the Restricted-access 
Group. When the Restricted-access Group switched to 


Finding 1 


the full ABCmouse curriculum in the second half of the 
school year, the two groups completed similar numbers 
of learning activities. Therefore, “regular usage” refers 
to both the access type and higher usage levels of the 
Full-access Group, and “limited usage” refers to both the 
limited access and lower usage levels of the Restricted- 
access Group. Students were tested at the beginning, 
middle, and end of the school year with the Classworks* 
and STAR® assessments, and with the DIBELS® at the end 
of the school year. 


Results 


The results from both studies demonstrate that ABCmouse 
helps accelerate academic growth, particularly literacy 
gains, in prekindergarten and kindergarten. The three 
main findings from these studies are summarized below. 


In Study 1, the more ABCmouse learning activities a student completed, the greater his or her kindergarten readiness 
score at the end of prekindergarten, indicating a reduction in the level of risk for school failure. 


Students who completed at least 35 activities (median activities completed) during prekindergarten demonstrated an 


additional 65% gain on the EPSF assessment than those who completed fewer than 35 activities, and there was no 
statistically significant difference in the students’ pretest scores administered before the prekindergarten year (Figure 1). 
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Figure 1. Mean Pre- and Posttest EPSF scores, grouped by the number of ABCmouse learning activities completed. Error bars represent +1 


standard error. *** t(202) = 3.39 on gain scores, p < .001, effect size d = .47. 


4 Classworks Reading and Classworks Math, Curriculum Advantage, Inc. 


5 STAR Math and STAR Reading, Renaissance Learning, Inc. 


6 Good, R. H., & Kaminski, R. A. (Eds.) (2007). Dynamic Indicators of Basic Early Literacy Skills (6th ed.). Eugene, OR: Institute for the Development of 


Educational Achievement. Available: http://dibels.uoregon.edu/. 
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Finding 2 


In Study 2, regular ABCmouse usage in kindergarten predicted students’ learning gains in literacy and math skills. 


While students in the Full-access Group started in Fall with slightly lower scores on the Classworks assessment than the 
Restricted-access Group, students in the Full-access Group demonstrated an additional 120% gain on early literacy skills 
from Fall to Winter, as compared to students in the Restricted-access Group (Figure 2). 
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Figure 2. Mean Classworks Reading Growth from Fall to Winter by access group in kindergarten. Error bars represent + 1 standard error. 
** (531) = 7.66, p < .001, effect size d = .66. The growth achieved by the Full-access Group was significantly higher than that of the 
Restricted-access Group when controlling for participation in the DPK program, F(1,531) = 29.00, p < .001, n? =.05. 
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Students in the Full-access Group also showed an additional 150% gain on early mathematics skills, compared to students 
in the Restricted-access Group (Figure 3). There was no group difference in Fall scores on early mathematics skills. 
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Figure 3. Mean Classworks Math Growth from Fall to Winter by access group in kindergarten. Error bars represent + 1 standard error. 
** (533) = 3.49, p < .001, effect size d = .30. The growth achieved by the Full-access Group was significantly higher than that of the 
Restricted-access Group when controlling for participation in the DPK program, F(1,530) = 59.64, p < .001, n? = 10. 


While students in the DPK program entered kindergarten at a higher readiness level on the STAR assessment in literacy 
and math than non-DPK students, with regular usage of the full ABCmouse curriculum in kindergarten (i.e., Full-access 
Group), non-DPK students were able to catch up with their DPK peers in both academic domains. Non-DPK students 
from both the Restricted-access Group and Full-access Group started out in Fall with significantly lower STAR Literacy 
scores than DPK students (p’s < .001, effect sizes were .45 for Restricted-access and .28 for Full-access), yet the non- 
DPK students with regular ABCmouse usage were able to close this gap and catch up to their DPK peers by Winter (p’s 
> 10 in Winter and Spring).’ The same patterns were seen with the STAR Early Numeracy scores (p’s < .001, and effect 
sizes were .48 for Restricted-access and .28 for Full-access in Fall; p’s > 10 in Winter and Spring).® 


By Spring, non-DPK students with regular usage demonstrated higher growth on early reading and math on the 
Classworks than non-DPK students with limited usage (Figures 4 and 5; p < .001, d = .42 for Reading and p < .05, d= 
.23 for Math). For DPK students, Spring scores were as high or higher than non-DPK students’ scores, but differences 
between DPK students’ Spring scores based on participation in the Full-access Group versus the Restricted-access 
Group were not statistically significant. 


7 In the Restricted-access Group, DPK students performed better than non-DPK students at all time points (p's < .001, d= .45 to .55). In the Full-access Group, 
non-DPK students had lower scores than the DPK students in Fall (p = .03, d = .28), but not in Winter and Spring (p’s > .10) 


8 In the Restricted-access Group, DPK students performed better than non-DPK students at all time points (p's < .001, d = .48 to .55). In the Full-access Group, 
non-DPK students had lower scores than the DPK students in Fall (p = .04, d = .28), but not in Winter and Spring (p’s > .10). 
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Figure 4. Classworks Reading score gain from Fall to Spring by participation in the DPK program and by ABCmouse access type. Error bars 
represent + 1 standard error. ***Non-DPK: Full-access vs. Restricted-access, t(326) = 3.82, p < .001, effect size d = .42; ***Restricted-access: DPK vs. 
Non-DPK, t(270) = 4.06, p < .001, effect size d= .50. 
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Figure 5. Classworks Math score gain from Fall to Spring by participation in the DPK program and by ABCmouse access type. Error bars represent 
+1 standard error. *Non-DPK: Full-access vs. Restricted-access, t(327) = 2.04, p = .04, effect size d = .23; ***Restricted-access: DPK vs. Non-DPK, 
t(272) = 4.06, p < .001, effect size d = .49; *Full-access: DPK vs. Non-DPK, t(257) = 2.24, p = .03, effect size d = .30. 
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Finding 3 
The number of ABCmouse learning activities completed over both school years was a strong predictor of literacy and 
math outcomes. 


ABCmouse usage over both school years reliably predicted students’ literacy and math performance at the end of 
kindergarten, after controlling for the effects of age, DPK enrollment, and ABCmouse access type received in 
kindergarten.® 


The multiple linear regression model on literacy outcomes predicts that for each 100 additional ABCmouse learning 
activities completed over both school years, students would realize an increase of .03 points on the DIBELS composite 
score, 9 points on the STAR Literacy, and 3 points on the Classworks Reading (Table 1). These are noteworthy increases, 
considering the range of total activities completed in both years was O to 5,784 activities (including repeats), and the 
ranges of Spring scores were -2.02 to 2.96 on DIBELS composite (z-scores), 127 to 1,655 on STAR Literacy, and 1,200 to 
1,330 on Classworks Reading. 


Literacy in Spring 


ae a E E 


DIBELS 10222 .0 Activities Completed*** .0003*** 
Age* 2i 1 
DPK Participation -.02 -.22 
Access Type*** 335 -.23° 

STAR Literacy SLO .06 Activities Completed*** {oygyr"* 21 
Age 6.85 .02 
DPK Participation** 83:651 12 
Access Type -9.56 -.03 

Classworks Reading 1412** 10 Activities Completed*** 1035 25 
Age* Swe) 09 
DPK Participation*** 13.46*** 17 
Access Type -.008 .0O 


Table 1. Multiple linear regression predicting each of Spring literacy and reading scores from total number of learning activities completed 
in prekindergarten and kindergarten, age, DPK participation, and access type. 


Note. DPK was coded as 7 and non-DPK as O; Restricted-access as 0, Full-access as 1. 
Statistically significant predictors are marked as follows: *** = p <.001, * = p < .01, * = p < .05. 


9 English Language Learning status and disability information were not available for students in the kindergarten sample. 


a The finding that having full access is negatively correlated with Spring DIBELS and STAR Literacy scores may be related at least in part to the children in the 
Restricted-access Group having started in Fall of kindergarten with higher scores on the Classworks Reading than children in the Full-access Group. 
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Similarly, the multiple linear regression model on math outcomes predicts that for each 100 additional learning activities 
completed, students would realize a .8-point increase on the STAR Early Numeracy and a 1.2-point increase on the 
Classworks Math (Table 2). The range of STAR Early Numeracy in Spring was 20 to 99, and the range of Classworks 
Math scores in Spring was 1,200 to 1,270. In most cases, DPK participation was also a statistically significant and positive 
predictor of Spring scores in literacy and math. 


Mathematics in Spring 


Ss ee a 


STAR Early Numeracy iy .0 Activities Completed** .008*** 
Age .89 105 
DPK Participation** 2.88** 12 
Access Type -.37 -.02 

Classworks Math 16.74*** 11 Activities Completed™™* OR .24 
Age*™* 4.53** 12 
DPK Participation*** 6.88*** .20 
Access Type .03 .001 


Table 2. Multiple linear regression predicting each of the Spring mathematics scores from total number of learning activities completed in 
prekindergarten and kindergarten, age, DPK participation, and access type. 


Note. DPK was coded as 1 and non-DPK as O; Restricted-access as 0, Full-access as 1. 
Statistically significant predictors are marked as follows: *** = p <.001, * = p < .01, * = p < .05. 


Conclusion 


This longitudinal study reveals that when young students complete more ABCmouse learning activities, their academic 
performance improves. The use of ABCmouse within the DPK program reduced the risk of school failure by the end 
of the prekindergarten year. This finding held constant during kindergarten. Furthermore, access to the full ABCmouse 
curriculum independently contributed to the academic growth of non-DPK students. The findings from these studies 
highlight the value of ABCmouse as a supplemental teaching resource in both prekindergarten and kindergarten, when 
students are learning the foundational skills in literacy and math that are essential to long-term success in school. 
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